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ABSTRACT 

Mobile Learning, or M-learning as it is often called, is a relatively new tool in the pedagogical arsenal to assist 
students and teachers as they navigate the options available in the expanding distance learning world. This 
article assesses some of the possible methods, challenges and future potential of using this approach in a college 
classroom and describes an empirical evaluation of the effectiveness of M-learning in a college classroom. One 
hundred twelve students in an introductory survey course in sociology were given the opportunity to use an M- 
Learning product developed by HotLava Software for the purpose of assisting them in preparation for two 
scheduled exams. Both practice and review questions were made available on Smart Phones, Web enabled 
phones, PDAs and other Internet capable mobile devices via Learning Mobile Author. Forty-two of the 112 
students in the class chose to access these data via their personal devices and their responses were collected and 
recorded. The results of their performance, as indicated by a final grade in the course, were compared to the 
outcomes for those students who chose not to use the M-Learning tool. Students using the software 
demonstrated a higher level of knowledge of the subject matter covered in the course when compared to 
students choosing not to use the tools (p<.01). Conclusions and a discussion of these outcomes are offered as 
well as some inferences and speculation regarding the future of M-Learning in the classroom and beyond. 
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Mobile Learning (M-Learning) is a relatively new tool in the pedagogical arsenal to support students and 
teachers as they navigate the options available in the expanding world of distance learning. M-Learning is 
learning accomplished with the use of small, portable computing devices. These computing devices may 
include: smartphones, personal digital assistants (PDAs) and similar handheld devices. M-learners typically 
view content and/or lessons in small, manageable formats that can be utilized when laptop or fixed station 
computers are unavailable. It is currently being used in a variety of educational, governmental and industrial 
settings. This paper assesses some of the possible methods; challenges and future potential of using this 
approach in a college classroom and provides an empirical evaluation of the effectiveness of this and similar 
tools. 

MOBILE LEARNING 

The first published studies focusing on M-Learning began around 2000. In the April 2000 issue of Computers 
and Education, Sharpies (2000) discussed the potential for new designs in personal mobile technologies that 
could enhance lifelong learning programs and continuing adult educational opportunities. Many, if not all, of 
the ideas raised in Sharpies’ early article are still evolving and are of interest to M-Learning today. 

Donna Abernathy’s article "Get Ready for M-Learning” (2001) provides one of the first looks at the technology 
and how it could affect future business approaches with regard to learning initiatives. This article may also be 
counted as an early alert about the potential to this already expanding learning option. Abernathy observes that 
M-Learning options don't necessarily seek to replace the PC as a be-all tool, but instead notes that it will help 
supplement corporate learning objectives with on-the-go tools. Businesses hope this will increase willingness 
among their employees and customers to interact with other learners and provide rapid feedback for corporate 
and sales force personnel. Although wifi technically existed at the time the Abernathy article was written, it was 
not fully developed or nearly as ubiquitous as it is currently. Abernathy accurately noted that the fledgling state 
of wifi technology could be a major stumbling block for future advances. 

Between 2002 and 2006 numerous studies began to appear that reported similar findings indicating M-Learning 
technologies were expanding and becoming commonplace in a variety of learning environments. 
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Seppala and Alamaki (2003) investigated the training and instruction of Finnish teachers using mobile 
technology in the classroom. Their experience and concerns with the new technology focused on several factors. 
First, they noted that, given that 98% of Finland’s university students owned cell phones in 2002, instruction via 
mobile learning opportunities seemed to be an important next step in the digital learning revolution. In their 
study of the use of Short Messaging Service text messaging (SMS) and digital pictures, content material was 
sent to a centralized memory bank. Each user could “withdraw” this material at any time for review and study. 
The teachers regarded the ability to take notes at any time and the capacity to work on materials during their 
daily travel time as a second advantageous feature. Feedback on educational content was offered almost 
instantaneously given the characteristics of the devices being used (cell phones, PDAs etc.), and the researchers 
felt that this factor allowed them to be more honest in their responses and opinions about the potential of mobile 
learning. Seppala and Alamaki concluded that M-Learning has a multitude of advantages, and that this 
technology has a place in the teaching models of the future. 

Attewell (2003) tackled a question many educators and some employers have had about mobile learning, 
particularly in relation to younger users. Some studies have questioned whether SMS “texting” could actually be 
harmful to a student's grammatical development (e.g. vocabulary acquisition and spelling) because text 
messages tend to be compact, and often times riddled with acronyms or purposely incorrect spelling designed 
for speed-of-input. Attewell agreed that these issues are valid concerns, and suggests more research into studies 
on mobile learning. Flowever she also notes how a classroom full of students with mobile devices came together 
to share content and messages, counteracting the assumption that cell phones have an isolationist effect. 

Other fields have begun inquiry into the effectiveness of mobile learning. Whitsed (2004) reviewed the advent 
of M-Learning and mobile computing in the field of medicine. The modern classroom environment for today's 
medical student is technologically sophisticated. Nevertheless PDA's that can access a patients charts from 
anywhere in a hospital provide a welcome alternative to having to log on to a networked terminal or a laptop 
just to recall patient details. Mobile devices also allow medical interns and residents to take notes and record 
audio, which can be studied and reviewed at a later date. The freedom afforded by being able to access 
information "anytime, anywhere" becomes a tremendous advantage and convenience when you take into 
account how many patients must be seen during a typical physician’s rounds. Whitsed states that 28% of US 
physicians already use mobile computing as part of their daily routine and that as this technology advances, this 
percentage will grow. 

Investigators have also examined how pervasive mobile technology is today and how it is still expanding. 
According to Wagner and Wilson (2005), cell phones now outnumber landline phones in America, and other 
wireless devices are gaining footholds with the help of workplace and community WiFi networks. The authors 
make an important distinction between M-Learning and E-Learning. They argue that as different devices and 
new delivery tools provide educators with far more options to reach today’s students, the education community 
must recognize that the model of “command and control,” typical of conventional education structures, is being 
replaced with a chance to make learning truly collaborative. 

Rushby (2005) explored M-Learning from a workplace perspective. He has compared the benefits of freedom of 
location with the traditional E-Learning models many companies have in place. His work suggests that M- 
Learning is superior to the older format particularly regarding employees’ ability to track and discover new 
knowledge in whatever setting best benefits their learning style. What has held back this type of educational 
openness was the limited memory and technology of past wireless devices. Now useful additions to mobile 
devices, such as GPS and video/audio enhancement make the possibilities of the devices themselves much more 
robust. The digitally versatile and collaborative nature of these new technologies is best captured by Jenkins' 
notion of media convergence, where “[A] whole range of new technologies enable consumers to archive, 
annotate, appropriate, and recirculate media content and in the process, these technologies have altered the ways 
that consumers interact with core institutions of government, education, and commerce” (Jenkins, 2006). 
Rushby believes that M-Learning is most likely to be adopted first by the business sector and that more research 
into it will only enhance this opportunity. As other risks and rewards are examined, other organizations such as 
high schools and universities should be able to pick up the technology and find an effective pedagogical use for 
it. 

Thornton and Houser (2005) have recently reported on their study of data and opinions regarding M-Learning 
and its use in a Japanese University. In Japan web enabled mobile phones, PDAs, and other portable media 
devices are extraordinarily prevalent and the populace is well versed in how to use them. In this study the 
investigators evaluated outcomes measures for classroom material using mobile phones, both via e-mail and 
using WAP technology (Wireless Application Protocol) for web enabled phones. It is important to note that in 
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Japan, rates for cell phone plans are far less expensive than in the United States, allowing more students to take 
part in M-Learning study and research. The results of their experiments were particularly revealing. The 
researchers observed improvements in test scores ranging from between 35% and 75%, using pre-post test 
measures over paper materials alone. The students’ reactions to this new learning opportunity were positive, 
although more so for the PDA trials than for the cell phone. The researcher’s noted that since the cell-phone is 
so well established in Japan, distributing educational material through them is a relatively effortless 
process. They stress that interactivity in content seemed a superior teaching method when compared to using 
static web pages alone for conveying material. As the United States catches up in more technologically 
advanced cell phone usage, M-Learning should become even more attractive to businesses and educational 
institutions wishing to advance their learning environments. 

Wagner (2005) has also reviewed the current state of mobile technology in America. Wagner cites examples in 
which mobile computing has made significant differences in peoples lives, such as helping to find survivors 
after the 2004 Indian Ocean tsunami. She notes that while mobile devices are yet to be the most prevalent tool 
available to many office workers and students, this technological landscape is changing and is becoming ever 
more friendly to M-Learning through the expansion of wireless networks and falling hardware and access 
prices. 

Lessons are to be learned from the implementation of E-Learning in the school environment as well, particularly 
with regard to the importance of varying content and interactivity. Wagner stresses that while M-Learning 
devices can be tremendously helpful, the information itself should be the focus of most of an educator’s 
attention, and that improving the content is the best approach to assuring that M-Learning is effective for all 
parties. In approaching the obstacles facing quality mobile learning experiences, Wagner concedes that 
ownership of the required hardware is the most important roadblock still standing in the way of widespread use. 
She goes on to note however that many universities are taking steps to correct this shortcoming by making the 
devices more available. By producing more quality content, regardless of the number of current users, students 
will be drawn to the flexibility and access of having a mobile device that can assist in their learning process. 

Corlett et al. (2005) report on a small class of students at the University of Birmingham who were given 
wireless PDA's for a semester to enable them to take part in an experiment on the possibilities and limitations of 
M-Learning in a college atmosphere. Students were familiarized with the capabilities and limitations of the 
hardware and then proceeded to use them for class work. The problems students reported to researchers were 
mostly technical in nature, for example having problems with limited memory of the handheld device or the 
battery life being too short. There were also some issues with the software used for the trial. At the end of the 
school year, discussion panels were held and a series of questionnaires were administered. The students’ 
experiences were mixed. The PDA had significant promise in their opinion, but they felt these devices did not 
live up to expectations. The hardware limitations of the devices used and software issues were enough for the 
students to offer somewhat lower ratings to M-Learning approaches than in some other studies. Nevertheless, 
the authors see these data as supporting information that can help improve upon the hardware and technology as 
these devices become increasingly ubiquitous in educational settings. 

M-Learning has shown increasing penetration at the professional level as well. As revealed by Pulchino (2006) 
in research published for the E-Learning Guild, collaboration of several different business organizations is 
underway. Pulchino sent questionnaires to a group of companies now using mobile computing to educate their 
workforce and to companies that were considering the option to upgrade from traditional computer assisted 
practices to mobile learning. He found that mobile devices were being used primarily for audio and text content 
and that wireless laptops were still the device of choice. However the iPod and cell phone were popular as well. 

Articles evaluating mobile learning technologies are increasing. Gomez (2007) has discussed how lessons and 
lectures delivered to students via mobile devices are evolving. He examined podcasts, audio and video files, and 
how they can be easily incorporated into courses. In general, the outcomes were rated very highly. After an 
initial period of training and acclimation, comments on how staff and students experience these opportunities 
were measured. His students reported that they enjoyed the ability to pause and segment lectures and to listen to 
them on their own time. An additional feature rated as highly valued centered on the fact that much of this 
information was readily available when students were away from their desktops. Gomez also reported that 
roadblocks do still exist however. Echoing Wagner (2005), he says that foremost among these issues is that all 
students do not necessarily have access to the required hardware (Gomez, 2007). 

At least one recent article has commented on a dimension to M-Learning that may seem contrary to its purpose. 
It has been noted that the content available in a designated M-Learning format may be accessed frequently on a 
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non-mobile device. Weekes (2007) suggests this is actually a strength of M-Learning. This is true because it 
validates mobile learning's place in the learning environment while demonstrating the importance of providing 
learners with choices about how and when they access learning materials. She goes on to indicate that, in theory 
at least, nothing liberates the learner more than mobile learning. 

The importance of M-Learning in some parts of the global knowledge enteiprise is obviously growing. And a 
quick search on LexisNexis (10-25-07) reveals more than one hundred related articles ranging from a news on 
recent M-Learning Conferences (such as the 1st Asia-Pacific Regional Mobile Learning Conference on August 
7, 2007 in Kuala Lumpur... see: www.lttcom.com/mlearning ) to stories on its popularity in Turkey, Korea, 
India, Nigeria. Thailand and many others. Very few articles have been published in the US, where awareness 
seems to be lagging. The authors suggest this is about to change as the US “catches up” with the developing 
world. 

AN EMPIRICAL EVALUATION 

In order to evaluate the effectiveness of M-Learning on student outcomes in the US higher education market, an 
empirical evaluation was undertaken. This study attempts to evaluate the effectiveness of using a relatively new 
mobile learning tool by HotLava Software to improve student performance. Essential to the expansion of these 
new mobile learning technologies and pedagogical approaches is the continuation of their empirical evaluations. 
It is hypothesized that employing mobile learning can impact performance (as measured by test scores) among 
college students. In this study, performance is measured by grade outcomes. Using HotLava WAP software 
called Learning Mobile Author (LMA), this study investigates the effects of access to review and study material 
made available on mobile devices-primarily web enabled cell phones. 

LEARNING MOBILE AUTHOR 

“Learning Mobile Author” is a mobile learning software program designed by Hot Lava Software Inc. It guides 
the user through the development and publishing of mobile-device-ready web content. The LMA software can 
be installed and run from most desktop machines with an Internet connection, and Hot Lava provide hosting 
services upon request. LMA also allows for the collection of details regarding individual user session activity 
and other connection information, effectively tracking time of use, session data, and any score or test 
information. The interface is simple, relatively intuitive, and allows for easy access to functions for revising and 
editing published web pages. LMA constructs pre-formatted XML code, and the content can be distributed in 
multiple file formats. 

METHODS 

The subjects for this experiment were students in a large Introduction to Sociology course at a midsized state 
university in the Northeast. The course was made up of primarily freshmen students taking the course to fulfill a 
general education requirement. The study was conducted over the course of the 2006 spring semester. There 
were a total of 112 students enrolled in the course (three students did not complete one of the scheduled tests 
prior to completing the final exam but were included in the data following a later make-up test). 

Two mid semester tests were given during the term at approximately the end of the first and second third of the 
course respectively. The average score for each student on the first two tests were used as the dependent variable 
to assess the effect of the independent variable: access to practice exam questions and vocabulary words via cell 
phone. These materials were also part of the weekly lectures and class reading assignments. Students were given 
the following instructions with regard to practice materials that would be made available to them in a variety of 
ways during the course of the semester: 

Throughout the semester you will be given access to vocabulary words and study questions of varying difficulty 
for review via your web-enabled cell phone. This material will be posted at various intervals prior to the two 
scheduled mid-semester exams. Access to these questions and terms can be accessed via the following URL 1 
and listed sources: 

• Via cell phone: http://www.WAP/xxx 

• Class review sessions 

• Class handouts 

You may use cell phone site and one or both of the classroom sources. The information from all sources is 
identical. 

The instructor then provided a detailed in-class explanation about how to access the ML A site. Additionally, 
specific review days when the classroom sources would be available were also designated. Students were then 
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allowed to choose a method by which they would retrieve the materials, and a password and ID were assigned 
using the cell phone technology. This introduces a potential source of bias. However, the investigators decided 
that a randomized approach would not allow for a determination of smart phone availability. This issue is 
discussed in more detail following the results section below. 

Four times prior to each mid semester exam, important people, practice questions and vocabulary words with 
definitions were placed on the WAP site in the form of multiple choice or T/F test items. These materials were 
also discussed and distributed on the specified review days prior to the tests as well. Table 1 provides examples 
that are illustrative of the kinds of material that were made available: 

Table 1: 

Examples of Practice Test Questions Published for Student Review 

1 . Mobility in a caste system is based on achieved status. True □ False □ 

2. If intergenerational mobility is high, stratification is relatively rigid. True □ False □ 

3. All deviance is bad for society. True □ False □ 

4. Deviance is related to social class. True □ False □ 

5. used the "term looking-glass self" to describe how a sense of self develops. 

a. Mead □ b. Cooley □ c. Goffman □ 

6. Pavlov, Watson and Skinner are associated with: 

a. Conflict □ b. Functionalism □ c. Behaviorism □ 

7. Who is this? 




a. August Comte □ 

b. Emile Durkheim □ 

c. Henri Saint-Simone □ 

Answers: l-T;2-F;3-F;4-T;5-b;6-c;7-a 



Students who accessed the MLA site were designated as experimental subjects and part of the experimental 
group. Those who used other methods to retrieve the review and study the practice information were identified 
as the control group. Each student was tracked over the semester while data on their use of one (or none) of the 
review methods were collected. Students who accessed or collected the materials on one or more occasions for 
each test were included as part of the study. After administering the two tests, the results were compiled and 
sorted into two groups: Group 1: The experimental group using the cell phone technology, n=42 and; Group 2: 
Students who did not access the site but collected material from the other sources one or more times n=70. 

The HotLava LMA software was employed to create the cell phone WAP web site. This software also 
automatically tracked who accessed the site, how often it was accessed, recorded their IDs and provided quiz 
scores feedback to the instructor and the student. 

RESULTS 

The purpose of this study was to evaluate the effectiveness of HotLava Software’s Learning Mobile Author in 
improving learning outcomes in an introductory undergraduate sociology class. Table 2 outlines the findings 
from the analysis. A statistically significant difference was found between the control and the experiment 
groups. A mean difference of 3 points and a median difference of 4 points were found between the two groups. 
The standard deviation for the control group was almost twice as high (11.7 compared to 6.1) as the 
experimental group. Those students who used the MLA/WAP web-enabled cells phones to retrieve and review 
the practice material provided for the two mid semester tests in the course had an average for the two tests of 
89% and those who used only the printed or discussion sources had an overall average of 84% (t=-2.50, p < .01). 
These findings indicate that students in this class using web-enabled cell phones to assist in their review of test 
materials out scored the students who used more traditional means (handouts and review lectures) to practice 
and review materials. 





The Turkish Online Journal of Educational Technology - TOJET July 2008 ISSN: 1303-6521 volume 7 Issue 3 Article 2 



Table 2: 

t Test for comparison of means between the experimental and control groups 



N=1 12 





Experimental Group 


Control Group 


n 


42 


70 


Mean 


89 


84 


95% Mean Conf 


87-91 


82-87 


SD 


6.1 


11.7 


High 


101 2 


101 2 


Low 


74 


38 


Interquartile Range 


98-86 


91-81 


Median 


89 


85 



T-value of difference: -2.498; df-t: 1 10 
Probability: 0.007757 (left tail pr: 0.99224) 

p < .01 



CONCLUSIONS AND DISCUSSION 

This study supports the notion that the use of mobile learning can make a positive and significant difference in 
the outcome performance as measured by average test score for students using the LMA software to review and 
practice for the exams. It should be noted that this is not meant to be a definitive study however. There are 
some important limitations to the investigation. As mentioned earlier, students who are willing to go the extra 
distance and become involved in a class by taking on novel study techniques may be more likely to succeed in 
the first place. It is possible that these same students would still have studied the quiz material just as fervently 
had it been distributed in any manner. But it is hard to argue against the idea that delivering information to 
mobile devices is at least a helpful convenience and probably much more. These results parallel the findings 
observed in other studies of mobile learning (Thornton and Houser, 2005) which have found increases in 
knowledge in students who were emailed study material to their cell phones compared to classmates who 
received hard-copy handouts of the same notes. 

Convenient availability of information and resources are strong bargaining chips in championing M-Learning. 
Instructors need to get the attention of their students to be effective educators, and with the advent of mobile 
devices there are even more opportunities to grab their imagination. As M-Learning obtains a foothold in 
American universities, class is theoretically always in session. The authors recommend that more college 
courses utilize a mobile-learning ready framework. If instructors begin experimenting with mobile learning, the 
field will experience a rapid growth in the quality and quantity of studies about the effectiveness of these tools 
and technique. The materials and knowledge required for an instructor to begin taking part in these new 
possibilities and pedagogies are becoming increasingly available to instructors. Many colleges around the world 
have begun this process. 

THE FUTURE OF M-LEARNING 

One important obstacle to overcome in the acceptance of M-Learning is that it is based upon students owning 
the necessary hardware. As noted, cell phone use among college students is pervasive, and trends have shown 
an ever-increasing saturation. The problem of outdated cell phones that cannot access the websites or materials 
directly will resolve itself. The realm beyond cell phones, however, is more difficult to analyze. While we know 
ownership of tools like the Blackberry or PDA's has risen, they have yet to permeate college campuses the way 
mobile phones have. This is echoed in the Corlett and Sharpies study where students reported enjoying having 
the school-loaned PDA for the class even though they would be unlikely to purchase the equipment themselves. 
The growth of WiFi networks in major metropolitan areas and increasing numbers of college campuses means 
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much of the backbone for the system may already be in place by the time schools and educators adopt these 
tools. "The adoption of next generation WiFi and MAN/WAN cellular networks will continue to deliver on the 
promise of ‘anywhere, anytime access’..." (Wagner & Wilson, 2005, p. 43). As new iPhone like devices 
become increasingly available and integrated with computers and desktop functions, the spread of M-Learning 
possibilities will certainly continue. 

The cell phone has become an ever-present fixture in the lives of modern college students, who use their devices 
to talk, message, capture pictures, and more recently-fo learn. Like remembering the keys to your home, 
students take their phones with them everywhere, and a large number of these devices are now web-enabled. 
These web-enabled phones, along with the increasingly popular PDAs and blackberry devices, can receive text, 
audio, and video information, as well as access web pages. It is doubtful that academia will let such a potentially 
transforming opportunity pass them by without taking advantage of this opportunity to deliver educational 
content. 

It is true that many college students have regular access to personal computers, delivering notes and study 
materials and even exams through course management systems it has taken more than a decade for this 
technology to reach the level it has. The personal computer may be a technologically more advanced medium, 
but it isn’t portable, and so it isn't something students are likely to carry around with them on a day-to-day 
basis. M-Learning does not seek to replace the utilization of computers to aid in learning, but rather to 
supplement it with interesting new methods that use a preferred medium increasingly available to students at 
affordable prices and already widely in use. 

Imagine a classroom where the instructor asks a question about subjects that have yet to be assigned or 
discussed in class. Instead of having to wait for an assignment or a lecture, they can use the devices at their 
disposal to access information themselves in a matter of seconds-all without leaving their desk. And, as was the 
case in this current study, practice material for an exam can be made available so students can truly study, and 
review and prepare for tests using a wide array of delivery options that will allow them to do this virtually 
anywhere at any time. 
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